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DETAILED ACTION 



1 . This communication is responsive to Amendment B, filed 4/21/04. 

2. Claims 1-21 and 29-30 are pending in this application. Claims 1-9 and 1 1-21 have been 
amended; claims 22-28 have been cancelled; and claims 29 and 30 have been added. 

3. The text of those sections of Title 35, U.S. Code not included in this action can be found 
in a prior Office action. 



4. The following is a quotation of the second paragraph of 35 U.S.C. 112: 

The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the 
subject matter which the applicant regards as his invention. 

Claims 11-16 rejected under 35 U.S.C. 1 12, second paragraph, as being indefinite for 
failing to particularly point out and distinctly claim the subject matter which applicant regards as 
the invention. 

5. Claim 1 1 recites the limitation "the pointing", "the display" and "the image" in line 2 of 
claim 1 1 . There is insufficient antecedent basis for this limitation in the claim. 



6. Claims 1-3, 6 and 10 are rejected under 35 U.S.C. 102(e) as being anticipated by Filo et 
al. ("Filo"). 

As per claim 1, Filo teaches a system for interacting v^ith displays and all devices that use 
such displays comprised of. 



Claim Rejections - 35 USC § 112 



Claim Rejections - 35 USC § 102 
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a) a display (col. 7, line 1 and lines 40-45); 

b) a pointing object (fig. Appointing object 72); 

c) a camera that has in its field of view at least one of the following: (i) the pointing 
object only, or (ii) the display and the pointing object, or (iii) the display and the reflection or 
effect that the pointing object can produce on the display (col. 9, lines 30-38 and lines 49-52; col. 
7, lines 19-22 and lines 51-54; col. 11, lines 32-55; col. 13, lines 34-35 and lines 10-13, 15-16; 
col. 14, lines 10-24); 

d) a method for detecting the position of the pointing object or its reflection or effect on 
the display in the image registered by the camera (col. 7, lines 18-20; col. 12, lines 26-34; col. 
14, lines 10-19); and 

e) a method for establishing the mapping between the position of the pointing object or 
its reflection or effect on the display in the image registered by the camera and a corresponding 
location on the display (fig. 10; col. 18, line 52 through col. 19, line 1; col. 7, lines 17-19; col. 
12, lines 26-34; col. 14, lines 10-19). 

As per claim 2, Filo teaches a system for interacting with displays and all devices that use 
such displays which commands the positioning of a pointing icon on the display (figs. 4B, 8, 10 
ad 11; col. 1 1, lines 32-55; col. 12, lines 26-35; col. 13, lines 34-35 and lines 10-13, 15-16; col. 
9, lines 49-50). 

As per claim 3, Filo teaches a system for interacting with displays and all devices that use 
such displays wherein the system commands the input of data into the device using the display 
(fig. 10; col. 18, line 52 through col. 19, line 1; col. 7, lines 17-19; col. 12, lines 26-34; col. 14, 
lines 10-19). 
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As per claim 6, Filo teaches a system for interacting with displays and all devices that use 
such displays wherein the pointing object is used to select, highlight, or define a particular point 
or region on the display (col. 9, line 61 through col. 10, line 3; col. 12, lines 26-35). 

As per claim 10, Filo teaches a system for interacting with displays and all devices which 
also includes at least one of the following (col. 9, line 49 through col. 10, line 22; col. 7, lines 40- 
49; col. 8, lines 10-21, 30-37; col. 1 1, lines 61-63; col. 13, lines 17-24; col. 14, lines 26-32): 

a) a method for selecting or highlighting a specific item or icon on the display; 

b) a method for activating a specific process, program, or menu item represented on the 
display; and 

c) a method for writing, scribing, drawing, highlighting, annotating, or otherwise 
producing marks on the display. 

7. Claims 11-16 are rejected under 35 U.S.C. 102(e) as being anticipated by Hansen. 

As per claim 1 1, Hansen teaches a method for detecting a position of a pointing object or 
its reflection or effect on a display in an image registered by a camera comprising the steps of: 

a) defining at least one characteristic of the pointing object or its reflection or effect on 
the display that (i) is registered in the image captured by the camera and (ii) distinguishes the 
pointing object from other objects registered in the image from the camera (col. 3, lines 28-40; 
col. 8, Hnes 30-49), 

b) retrieving the image from the camera (col. 8, lines 12-20), 

c) analyzing the image from the camera to locate the characteristic or characteristics of 
the pointing object or its reflection or effect on the display pointing object (col. 8, lines 12-50), 
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d) determining the most likely position of the pointing object or its reflection or effect on 
the display in the image from the camera based on at least one of the following: (i) the last 
known position of the pointing object or its reflection or effect on the display in the image, (ii) 
the position or positions at which the at least one distinguishing characteristic of the pointing 
object, or its reflection or effect on the display or the set of the picture elements in the image that 
comprise the rendition of the pointing object, or its reflection or effect on the display (col. 4, 
lines 27-31; col. 7, lines 18-27; col. 8, lines 40-45; col. 9, lines 3-25). 

As per claim 12, Hansen teaches a method for detecting a position of a pointing object or 
its reflection or effect on a display in an image registered by a camera wherein at least one 
characteristic that distinguishes the pointing object from other objects in the image from the 
camera is known a priori (col. 4, lines 55-56). 

As per claims 13 and 14, Hansen teaches a method for detecting a position of a pointing 
object or its reflection or effect on a display in an image registered by a camera wherein at least 
one characteristic that distinguishes the pointing object from other objects in the image from the 
camera are determined based on analysis of at least one set of the data acquired from the sensor 
or one image acquired from the camera and whose rendition are present in the image from the 
camera is obtained by acquiring at least two images from the camera, one with the pointing 
object in view of the camera and one without, and comparing the two sets with one another (col. 
4, Hnes 15-41; col. 7, lines 18-27; col. 8, lines 38-45). 

As per claim 15, Hansen teaches a method for detecting a position of a pointing object or 
its reflection or effect on a display in an image registered by a camera wherein adjustments or 
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modifications are made to the position, viewing angles, sensitivity, and other settings of the 
camera pursuant the analysis of the data or image retrieved firom the camera (col. 4, lines 12-18). 

As per claim 16, Filo teaches a method for detecting a position of a pointing object or its 
reflection or effect on a display in an image registered by a camera wherein at least part of the 
procedures for the method is carried out using at least in part the computing mechanism available 
on one or more of the following: the display, or the camera, or the pointing device, or the device 
producing the signal shown on the display, or the device producing the pointing icon on the 
display (fig. 1; col. 2, line 66 through col. 3, line 55). 

Claim Rejections - 35 USC § 103 

8. Claim 8 is rejected under 35 U.S.C. 103(a) as being unpatentable over Filo in view of 
Hansen. 

As per claim 8, although Filo teaches a system for interacting with displays and all 
devices that use such displays wherein the pointing icon on the display can be registered by the 
sensor (figs. 2B, 8, 10 and 1 1; col. 11, lines 43-46; col. 12, lines 31-32; the sensor having 
camera capabilities, i.e, by definition, a camera is the part of a television transmitting apparatus 
that receives the primary image and transforms it into electrical impulses), Filo does not 
explicitly disclose the pointing icon being registered by the camera. Hansen teaches a pointing 
icon being registered by a camera (col, 4, lines 27-31; col. 7, lines 18-27; col. 8, lines 40-45; col. 

9, lines 3-25). Therefore, it would have been obvious to an artisan at the time of the invention to 
include Hansen's pointing icon being registered by a camera to Filo's pointing icon being 



Application/Control Number: 09/757,930 Page 7 

Art Unit: 2174 

registered by a sensor in order to provide users with an alternate and/or a greater variety of signal 
receiving devices. 

9. Claims 17-21, 29 and 30 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Hansen in view of Filo. 

As per claims 17 and 29, Hansen teaches a method for establishing a mapping between 
the set of positions that a pointing object can assume in addressing a set of corresponding points 
or regions on the display comprising the steps of defining the boundaries of the positions that the 
pointing object can assume in addressing points or regions on the display and defining the 
boundaries of the display (Abstract; col. 3, Hnes 21-42; col. 3, line 63 through col. 4, line 8), 
defining the boundaries of the display and defining within the boundaries of the display, a 
continuous pointing object surface (Abstract; col. 3, Hnes 21-42; col. 3, line 63 through col. 4, 
line 8), warping the geometry of the surface for the pointing object so that at least one region of 
the pointing object surface overlaps with at least one region of the display surface and 
establishing a one-to-one or many to one correspondence between overlapping the said region of 
the pointing object surface and display surface, respectively (col. 7, lines 13-20; the geometry of 
the pointing object surface is scaled or warped so that regions of the pointing object surface 
have an area in common with/overlaps with a region of the display surface to establish a one-to- 
one or many to one correspondence between regions of the pointing object surface and display 
surface). However, Hansen does not explicitly disclose segmenting the display surface and 
pointing object surface into at least two regions. Filo teaches a method for establishing a 
mapping between the set of positions that a pointing object can assiune in addressing a set of 
corresponding points or regions on the display comprising segmenting the display surface and 
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pointing object surface into at least two regions (col. 3, lines 38-42; col. 12, lines 26-35; col. 14, 
lines 10-24; segments of the display corresponds with segments of pointing object surface 
selected and wherein detecting a display perspective is inherent when there is already a method 
for detecting the pointing device in an interactive virtual world environment) and warping the 
geometry of the surface for the pointing object so that at least one region of the pointing object 
surface overlaps with at least one region of the display surface to establish a one-to-one or many 
to one correspondence between overlapping the said region of the pointing object surface and 
display surface (col. 17, line 33 through col. 18, line 24; col. 18, line 52 through col. 19, line 5). 
Therefore, it would have been obvious to an artisan at the time of the invention to include Filo's 
method for establishing a mapping between the set of positions that a pointing object can assume 
in addressing a set of corresponding points or regions on the display comprising segmenting the 
display surface and pointing object surface into at least two regions to Hansen's method for 
establishing a mapping between the set of positions that a pointing object can assume in 
addressing a set of corresponding points or regions on the display in order to provide users with 
the ability to compartmentalize or organize areas of focus according to content. 

As per claim 18, the modified Hansen teaches a method for establishing a mapping 
between the set of positions that a pointing object can assume in addressing a set of 
corresponding points or regions on the display wherein the boundaries of the set of positions that 
the pointing object can assume are obtained by querying the user to point to the boundaries 
(Hansen: Abstract; col. 3, lines 21-42; col. 3, line 63 through col. 4, line 8; Filo: col. 12, lines 5- 
22; col. 12, hnes 26-35; col. 14, lines 10-24). 
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As per claim 19, the modified Hansen teaches a method for establishing a mapping 
between the set of positions that a pointing object can assume in addressing a set of 
corresponding points or regions on the display wherein the boundaries of the set of positions that 
the pointing object can assume are obtained by contours or the periphery of the display surface as 
the display is present in the image from the camera (Hansen: Abstract; fig, 1; Filo: col. 12, lines 
5-22), 

As per claim 20, the modified Hansen teaches a method for establishing a mapping 
between the set of positions that a pointing object can assume in addressing a set of 
corresponding points or regions on the display wherein the regions of the display surface 
comprise at least two sets of pixel elements that comprise the image on the display (Hansen: col. 
5, lines 1-15; Filo: col. 12, lines 5-22). 

As per claim 21, the modified Hansen teaches a method for establishing a mapping 
between the set of positions that a pointing object can assume in addressing a set of 
corresponding points on the display wherein at least part of the procedures for the method is 
carried out using at least in part the computing mechanism available on one or more of the 
following: the display, or the camera, or the pointing object, or the device producing the signal 
shown on the display, or the device producing the pointing icon on the display (Hansen: fig. 1; 
col. 2, line 66 through col. 3, line 55; Filo: figs. 1 and 2; col. 6, lines 6-66). 

As per claim 30, the modified Hansen teaches a method for establishing a mapping 
between the regions of a display and an image of the same display obtained fi*om a camera 
wherein at least one special image is rendered on the display for establishmg a mapping between 
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the regions of a display and an image of the same display obtained from a camera (Filo: figs. 2B, 
8, 10 and 1 1; coL 1 1, lines 43-46; col. 12, lines 31-32; col. 18, line 52 through col 19, line 5), 

10. Claim 7 is rejected under 35 U.S.C. 103(a) as being unpatentable over Filo et al. ("Filo") 
in view of Edwards et al. ("Edwards"). 

As per claim 7, Filo teaches a system for interacting with displays and all devices that use 
such displays comprising a pointing object for entering information into the system and can be 
used in conjunction with various software appHcations (col. 7, lines 46-54; col. 8, lines 16-35; 
col. 10, lines 24-31 and line 57; col. 13, lines 21-25; information entered into the system further 
include a clipboard on which a user can draw). Filo does not explicitly disclose the pointing 
object to be used to define a vector on the plane of the display that indicates a direction and 
magnitude relative to or with respect to an item on the display or a region of the display. 
Edwards teaches a system for interacting with displays wherein the pointing object to be used to 
define a vector on the plane of the display that indicates a direction and magnitude relative to or 
with respect to an item on the display or a region of the display (col. 8, lines 12-21). Therefore, it 
would have been obvious to an artisan at the time of the invention to include Edwards' teaching 
of a computer aided design tool, wherein the pointing object to be used to define a vector on the 
plane of the display that indicates a direction and magnitude relative to or with respect to an item 
on the display or a region of the display, to Filo's teaching of a system for interacting with 
displays that includes drawing capabilities to provide users with an environment capable of 
incorporating multiple applications and capabilities to enhance a user's individual task. 

11. Claim 9 is rejected under 35 U.S.C. 103(a) as being unpatentable over Filo et al. ("Filo", 
US 6,215,498 Bl) in view of Applicant's Admitted prior art. 
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As per claim 9, although Filo teaches a system for interacting with displays and all 
devices that use such displays which also includes a method for sensing the pointing object's 
position relative to the position of a pointer icon on the display (col. 7, lines 1-19), Filo does not 
explicitly disclose the system to include a method for correcting the offsets between the position 
of the pointing object, or reflection, or effect thereof on the display as observed by the user or by 
the camera, and the position of the pointer icon on the display. However, Applicant's admitted 
prior art teaches a system for interacting with displays and all devices that use such displays to 
include a method for correcting the offsets between the position of the pointing object, or 
reflection, or effect thereof on the display as observed by the user or by the sensor or the camera, 
and the position of the pointer icon on the display (page 21, lines 6-9). Therefore, it would have 
been obvious to an artisan at the time of the invention to include Applicant admitted prior art's 
teaching of a system for correcting the offsets between the position of the pointing object and the 
position of the pointer icon on the display to Filo's method of a system for sensing the pointing 
object's position relative to the position of a pointer icon on the display in order to reduce the 
margin of error concerning the position of the pointing object and the position of the pointer icon 
on the display. 

12. Claim 4-5 are rejected under 35 U.S.C. 103(a) as being unpatentable over Filo et al. 
("Filo", US 6,215,498 81). 

As per claim 4, although Filo teaches a system for interacting with displays and all 
devices that use such displays wherein the pointing object is a part of the body of a user of the 
system such as the hand or the finger, or an ornament or an object or device worn on the body 
such as a glove or thimble, Filo does not explicitly disclose the camera being remote fi-om the 
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human body (figs. 2(A-B), 4, 5(A-B) and 1 1). Official Notice is given that a camera being 
remote from the pointing object is well known in the art. Therefore, it would have been obvious 
to an artisan at the time of the invention to include sensors or camera being remote from the 
human body to Filo's. teaching of sensors or cameras in order to provide users with an 
implementation preference. 

As per claim 5, Filo teaches a system for interacting with displays and all devices that use 
such displays wherein the pointing object is used to point to regions of the display by way of 
changing its position, attitude, or presentation (col. 9, line 61 through col. 10, line 3). 



Response to Arguments 
13. Applicant's arguments with respect to claims have been considered but are moot in view 
of the new ground(s) of rejection, except for the following: 

Filo does not show or disclose a method for detecting the position of the pointing object 
or its reflection or effect on the display in the image registered by the camera wherein the 
reflection or the effect on the display covers a laser pointer shining on a projected screen. 

The Examiner disagrees for the following reasons: 

In response to applicant's argument that the references fail to show certain features of 
applicant's invention, it is noted that the features upon which applicant relies (i.e., the reflection 
or the effect on the display covers a laser pointer shining on a projected screen) are not recited 
in the rejected claim(s). Although the claims are interpreted in light of the specification, 
limitations from the specification are not read into the claims. See In re Van Geuns, 988 
F.2d 1 181, 26 USPQ2d 1057 (Fed. Cir. 1993). However, Filo does teach a method for detecting 
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the position of the pomting object or its reflection or effect on the display in the image registered 
by the camera (col. 7, lines 19-20). 



14, Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Examiner Le whose telephone number is (703) 305-7601. The 
examiner can normally be reached on Monday - Friday from 5:30 am to 2:00 pm (EST). 



supervisor, Kristine Kincaid, can be reached on (703) 308-0640. 

The fax numbers for the organization where this application or proceeding is assigned are 
as follows: 

(703) 872-9306 [Official Communication] 

Any inquiry of a general nature or relating to the status of this application or proceeding 
should be directed to the receptionist whose telephone number is (703) 305-3900. 



Inquires 



If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
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Patent Examiner 
June 7, 2004 
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